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NASA’s human exploration initiatives will invest in technologies, public/private 
partnerships, and infrastructure, paving the way for the expansion of human civilization 
into the solar system and beyond. As it is has been for the past half century, the Kennedy 
Space Center will be the embarkation point for humankind’s journey into the cosmos. 
Functioning as a next generation space launch complex, Kennedy’s launch pads, integration 
facilities, processing areas, launch and recovery ranges will bustle with the activities of the 
world’s space transportation providers. In developing this complex, KSC teams work 
through the potential operational scenarios: conducting trade studies, planning and 
budgeting for expensive and limited resources, and simulating alternative operational 
schemes. Numerous tools, among them discrete event simulation (DES), were matured 
during the Constellation Program to conduct such analyses with the purpose of optimizing 
the launch complex for maximum efficiency, safety, and flexibility while minimizing life 
cycle costs. Discrete event simulation is a computer-based modeling technique for complex 
and dynamic systems where the state of the system changes at discrete points in time and 
whose inputs may include random variables. DES is used to assess timelines and throughput, 
and to support operability studies and contingency analyses. It is applicable to any space 
launch campaign and informs decision-makers of the effects of varying numbers of 
expensive resources and the impact of off nominal scenarios on measures of performance. In 
order to develop representative DES models, methods were adopted, exploited, or created to 
extend traditional uses of DES. The Delphi method was adopted and utilized for task 
duration estimation. DES software was exploited for probabilistic event variation. A roll-up 
process was used, which was developed to reuse models and model elements in other less- 
detailed models. The DES team continues to innovate and expand DES capabilities to 
address KSC’s planning needs. 


Nomenclature 

a = alpha, Type I error, Producer’s risk 

p - beta, Type II error, Consumer’s risk 

Hq = null hypothesis 
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N(x,s) = normal distribution, mean x and standard deviation s 
p 0 = desired proportion that passes 

Pj = proportion of sample that passes 

I. Introduction 

A t NASA’s Kennedy Space Center, ongoing planning has been underway for the ground processing and 
integration of human-rated launch vehicles and spacecraft, most recently for the Orion Spacecraft, the Altair 
Spacecraft and the Ares launch vehicles and moving forward for private/public exploration initiatives. The planning 
involves identification, development, modification, and risk assessment of facilities, ground processing activities, 
ground support equipment, and budgeting thereof. Additionally, the planning work informs flight hardware design 
teams on how to improve operability and maintainability of the flight hardware. Among the many tools KSC uses is 
discrete event simulation (DES), the subject of this paper. 

Discrete event simulation is a computer-based modeling technique for complex and dynamic systems whose 
state changes at discrete points in time and whose inputs may be random variables. In a DES model, activity 
networks are implemented, task durations are fitted with theoretical or empirical distributions, and constrained 
resources are allocated across applicable tasks. Models are executed on computers and the random variable outputs 
are analyzed and interpreted. 

This paper provides a brief overview of DES, how it has been used at KSC, and the benefits it is providing to 
KSC in its role as a hub for human exploration. We then delve into the description of methods developed and 
exploited to support the DES analyses for GOP, which may be applied elsewhere, both in aerospace and generally in 
DES. Among these are an implementation of the Delphi method, an approach to probabilistic event variation, and an 
overview of how the team rolls up detailed results to be used in less-detailed models to support decision making. 

II. Discrete Event Simulation at Kennedy Space Center 

In this section we provide an overview of DES generally. Then we describe how DES has been used at the 
Kennedy Space Center along with a description of the benefits to the Center. 

A. Overview of DES 

Discrete event simulation (DES) is a method of modeling system behavior under the assumption that the state of 
the system — described by a collection of variables — changes at discrete points in time , 1 The computational 
techniques can be implemented within a computer program via an event scheduling approach that is very efficient, 
which allows one to model complex systems. There are several commercially available software products that 
embody DES techniques. We use Arena® software from Rockwell Automation® to provide the DES environment 
for developing models and providing analysis at NASA’s Kennedy Space Center. A significant benefit of 
simulation, modeling and analysis is that it allows one to conduct experiments and make predictions about system 
behavior prior to building or modifying a complex system such as a space launch vehicle. 

A project such as building a house or preparing a rocket for launch can be modeled with DES and these models 
can be used to provide valuable information on the project. For example, the likely time to complete the house or 
get the rocket ready for launch can be estimated with the model. While a simple project schedule could be used by 
itself to estimate the planned time to build the house or complete the rocket, the simulation model allows one to 
include uncertainty factors such as resource availability and variability in how long each individual step in the 
project will take. Running the model once, called a replication, simulates the building of the house or preparing the 
rocket from start to finish. Many replications of the model are run to determine a probability density function for the 
time to complete the project e.g., build the house or prepare the rocket. Since each replication will experience 
different inputs, through the use of different random numbers for process delays, there will be variability in the 
project duration for each replication. The benefit of this is that we can simulate preparing a rocket for launch with 
different assumptions so that we can determine the quickest or optimum way in which to prepare the rocket. 

Keys to modeling complex systems and providing useful information to decisions makers include: developing 
good problem statements; working closely with the decision makers; verifying that your model is behaving the way 
it is intended; performing statistically valid input and output analysis; and validating that the model represents the 
system under study with sufficient accuracy to make decisions . 2 
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B. DES at Kennedy Space Center 

Discrete-event simulation models and the resulting analyses have been utilized by decision makers at KSC for a 
. number of years. DES expertise and experience at KSC has been matured greatly, this section provides a brief 
history of those efforts. 

In the 1993-1994 timeframe, a space shuttle Launch Processing Model was developed. 3 This model was used to 
predict annual flight rates and could also determine how many 8 -hour shifts would be required to perform orbiter 
processing prior to readiness of orbiter mate to the external tank. 

In 1999, KSC entered into a Space Act agreement with the University of Central Florida to build a macro-level 
model of the space shuttle including manufacturing, refurbishment, ground processing, launch, ascent, on-orbit 
operations, return to earth, and recovery. 4 This project was highly successful in developing a valid model and 
developing a cadre of experts to provide future DES modeling and analysis expertise to NASA. 

In 2003, KSC funded the development of a DES tool called the Generic Environment for Modeling Future 
Launch Operations, GEM-FLO. GEM-FLO is a generic discrete event simulation tool that facilitates the creation of 
simulation models for modeling ground processing. GEM-FLO accepts launch vehicle design characteristics and 
operational inputs from the user, and uses these to configure a simulation model that properly reflects the ground 
processing flow and requirements of that specific launch vehicle. When the simulation model is executed, it 
provides a number of measures of performance including operations turnaround time, expected flight rate, and 
resource utilizations, thus enabling users to fairly assess multiple future vehicle designs using the same generic tool. 
GEM-FLO was used by the Next Generation Launch Technology (NGLT) Program and the Orbital Space Plane 
(OSP) Program. 5 ' 6 

DES models built specifically for the space shuttle program have been used at KSC to determine how long it 
would take to complete a sequence of space shuttle launches and how many space shuttle launches could be 
achieved over a specified period of time. 7,8,9 

In 2008, the Constellation Ground Operations Project at KSC setup a Discrete Event Simulation Modeling & 
Analysis Center to analyze and evaluate the post-Shuttle ground processing operations at KSC. Since that time, 
models have been developed and extensive analyses have been performed for the majority of the major processing 
facilities at KSC. This DES capability has been leveraged to address a number of critical questions for the future of 
KSC ground operations such as flight rate capabilities, resource requirements, facility layout and design, and launch 
probability. 

C. Benefits of DES for NASA’s Exploration Initiatives 

The Constellation program was established with a goal of substantially reducing the cost of human spaceflight in 
order to facilitate sustainable exploration operations in conjunction with the development of new exploration 
systems. Lessons learned from the Space Shuttle Program and International Space Station Program prescribe that 
program operations costs are unaffordable if lifecycle cost implications of decisions made during development are 
ignored or poorly understood. Indeed, the gross majority of life cycle costs are incurred in the operational phase of 
large, long-term programs such as Constellation. Determined to meet the objective of sustainable cost for ground 
processing and launch operations of the Constellation program, the Ground Operations Project at KSC established a 
systems engineering approach to operations planning, in essence, “designing the operations of the systems” in 
conjunction with designing the Constellation systems themselves. This approach provides a key feedback loop that 
allows systems designers and program management to understand the operational and ultimately lifecycle cost 
implications of their system design decisions. A critical enabling tool to this approach is DES. In much the same 
sense that modem CAD software allows designers to model the structures and mechanisms of a hardware system, 
DES allows the operations team to model all of the processes, interdependencies, and uncertainties of a projected 
processing flow, and evaluate the response to key operability metrics. 

An early application of DES in the Constellation Program was an analysis of the preferred layout of the Vehicle 
Assembly Building (VAB), where the Ares I and Ares V rockets would be stacked and tested on their Mobile 
Launchers. The VAB has four large integration bays, each capable of supporting the stacking of a vehicle. 
However, the bays share cranes, transfer aisle space, and have other key interdependencies that drive complex 
interactions when multiple vehicles are integrated at once, as would be the case for supporting lunar and eventually 
Mars exploration missions. A DES was constructed that modeled all of the stacking and integration processes in the 
VAB, including seizure of key resources and wait time due to operational clears associated with hazardous 
operations, and several different launch campaign scenarios were evaluated in the simulation. Ultimately, certain 
layouts proved to be considerably more efficient and a layout was chosen. 

Another application of DES was in Constellation’s requirements formulation period. KSC supported analysis 
led by the Langley Research Center to determine the optimum launch spacing and order between the Ares I, which 
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would carry the crew for the lunar mission, and the Ares V, which would carry their lunar lander. The DES 
modeled the launch processing flow of the two vehicles, accounting for system reliability, weather, abort zone 
implications, the Low Earth Orbit loiter capabilities of the spacecraft and propulsion stages, and other key drivers to- 
determine the relative advantages of the different possible launch orders and separation. Considering the risk 
posture and potential outcomes that the simulation predicted, the program elected to specify an aggressive 90 minute 
launch separation and elected to protect for launching either vehicle first, depending on the specifics of the 
mission. 10,11 

In support of the Ground Operations Preliminary Design Review, DES has been used extensively to analyze 
compliance of the Constellation Systems with respect to several key system requirements, including launch rate, 
launch spacing, and vehicle processing operational timelines. DES was also used to confirm that the numbers of 
resources required to process Ares I and Orion are within the budget levels allocated. 

Perhaps the most important applications of DES were in validating Ground Operations “single-string” processing 
architecture. The Apollo and Space Shuttle programs both utilized multiple copies of major ground systems 
infrastructure elements, such as launch pads, mobile launch platforms, Launch Control Center firing rooms, and 
VAB high bays. While this approach provided flexibility and capability, it was also very costly to maintain. In order 
to substantially lower launch site operations costs, the Ground Operations Project boldly proposed a single-string 
processing architecture, whereby only single units of major infrastructure elements would be developed for Ares 
I/Orion. DES was used to validate that the critical path processing flows for Ares and Orion were kept in check, and 
Ground Operations could indeed meet the launch manifest required by the program with acceptable risk. 

In part due to state-of-the-art operations planning tools such as DES, as of PDR, it is projected that the Ground 
Operations cost for the Ares I/Orion will be considerably less than the Space Shuttle or Apollo, meeting the goal set 
for the Constellation Program. As KSC’s role as the nation’s embarkation point for humankind’s journey into the 
cosmos evolves, DES will continue to play a critical role in optimizing the operations of the launch complex. 

III. Discrete Event Simulation Extensions 

Planning the ground and launch processing at KSC to optimize facility space, resources, ground support 
equipment, and personnel, has been the focus of the DES Modeling & Analysis Center. The center has developed 
and exploited a variety of methods and tools to support KSC’s launch ‘technology and development efforts to 
continue its role as a world-class hub for human spaceflight. In this section we describe three such methods, the 
Delphi method for input distributions, probabilistic event variation, and a level-of-detail roll up method. 

A. Delphi Method 

Immediately after the creation in 2008 of the DES Modeling & Analysis Center at Kennedy Space Center, DES 
modeling efforts were undertaken to describe and experiment with the planned ground processing flow of the Orion 
Spacecraft. An activity network of the flow had been developed, but, because the Orion Spacecraft is new, no 
relevant historical data were available for the task durations used in the DES model. Therefore a method for 
generating task duration estimates in the absence of historical data was needed; the team proposed the use of the 
Delphi Method for this purpose. 

1. Common Cause Variability in Task Durations 

One of the major benefits of DES modeling is that the task durations are modeled not as deterministic values but 
rather as variable quantities that may take on a range of values. Figure la illustrates the use of a deterministic point 
estimate of a task-duration. Deterministic point estimates equate to statements such as this: “100% of the times this 
process is done, it will take 4 hours”. DES is exploited when task durations are modeled with a distribution, such as 
Figure lb. Statements may be made such as “the task will take between 2 and 8 hours”, or “the task will take 4 hours 
most of the time”, or “70% of the times this task is executed its duration will be between 3 and 5 hours”. Such 
distributions account for the uncertainty due to “common cause” variability, which may be from seasonal factors, 
day-o f-the- week factors, minor rework, minor adjustments, cure time, and so forth. These distributions do not 
account for “special cause” variability, which may be due to learning curves, unplanned work, and contingencies. 
Special cause events may be modeled in DES as a branch to a contingency logic flow. 

Because task duration distributions are preferable to deterministic point estimates the DES team chose to use the 
Delphi Method to collect task duration estimates. 
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Figure 1. Task Durations: a) Point Estimate b) Distribution Estimate 


2. Reasons for using the Delphi Method 

The Delphi method is a structured elicitation method for forecasting, estimating, and planning used with a group 
of subject matter experts (SMEs) in the absence of sufficient data. The team employs the Delphi method in 
estimating input quantities used in process models. 

The Delphi Method 
was originally developed 
and introduced by the 
Rand Corporation in the 
1950s and first discussed 
in Dalkey and Helmer 
technical reports. 12 The 
genesis of the method 
was Project Delphi 
sponsored by the U.S. 

Air Force to obtain the 
most reliable consensus 
of opinion among a 
group of experts in 
enemy targeting 

decisions. It has been 
used for highly complex 
issues beyond 

forecasting such as waste water planning, drug abuse policies, plastics industry forecasting, and the future of the 
steel industry. An overview of the method follows. First several subject matter experts (SMEs) are each individually 
given a task of providing an estimate, forecast, or viewpoint on a specific question. Their responses are kept 
anonymous. The collection of responses are tallied, quantified, or otherwise assembled by a facilitator. The 
anonymous summaries are sent back to the group of SMEs for revision in a second round, if needed. SMEs respond 
again, providing their opinion again, with consideration of the anonymous feedback of their peer SMEs from the 
prior rounds. Within 2 or 3 rounds, consensus is formed. 

There are numerous benefits to the Delphi method over other group elicitation methods. First, the SMEs have 
anonymity unlike group meetings. This avoids dominance of a few individuals in the group setting. All SMEs get 
equal time. Conformance of opinion is avoided since opinions are provided privately. This approach avoids 
competition among SMEs and embarrassment among SMEs for unpopular views. 

Because of these features, better opinions and forecasts are reached. This is due to the time for thinking and 
reflecting by SMEs and the flexibility to participate at one’s convenience. Broader ideas are often found and there is 
a lower probability of fixation on one idea. Individuals focus on the problem rather than on how they present 
themselves in a group meeting. 1 ’ 

The Delphi method also provides benefits to the organization. The time impact on SMEs is minimized, 
scheduling conflicts of a group meeting are avoided, and there are neither travel expenses nor time away for group 
meetings in other cities. Team interactions are positive, too. 

Finally there are benefits of this approach to the DES models. The method has been shown to be a scientifically 
valid approach of estimating and forecasting in the absence of existing data and provided well demonstrated results 
in its past use. It is less intrusive, more objective, and a more reliable approach for the subjective estimation of task 
durations. Convergence to unbiased, independent, knowledge-representative estimates for task durations result from 
its use. 14 


3. Input Data Collection and Analysis Approach 

For DES models, the list of tasks needing task duration estimates, along with the work breakdown structure 
(WBS) code for the task, columns for entering the minimum task duration, the most likely task duration, and the 95 th 
percentile (near-maximum time) task duration, a column for entering the rationale for a task duration estimate, and 
columns with the names of SMEs with expertise for the task were assembled in a Microsoft® Excel® spreadsheet. 
This spreadsheet was then distributed to a list of SMEs generated by the team. The SMEs were provided instruction, 
both written and via a workshop, which included the illustration in Figure 2, on how to complete the spreadsheet 
along with the context of the tasks that they were estimating, specifically various conceptual models of the process 
being considered. 
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The SMEs completed the spreadsheets, providing minimum, most likely, and 95 th percentile task duration 
estimates on the specific tasks for which they have expertise. Some SMEs had only a few tasks while other SMEs 
had hundreds of tasks to estimate. 

The spreadsheets were returned to a 
DES analyst. The task time data were 
analyzed for convergence, and if 
needed specific tasks were 
distributed again to SMEs for 
revision. Once converged, the 
median response among the set of 
responses for each of the three 
elements of the triad was computed. 

The triads of median results were 
then used to fit theoretical continuous 
distributions using ExpertFit®. 

ExpertFit may be used to analyze historical data when they are available and are accurate, or in this case, 
ExpertFit may be used on Delphi Method outputs. The Delphi results (minimum, most likely, and 95 th percentile) 
were used as inputs in ExpertFit to generate a curve, as illustrated in Figure 2. ExpertFit is distributed by Averill M. 
Law & Associates. According to the User’s Guide, 15 

“ExpertFit will automatically and accurately determine which probability distribution best represents your data set, and 
do so in a few seconds. Its distribution-selection algorithm was developed by evaluating 15 heuristics on 35,000 data 
sets, with the analysis of the data taking six months. ExpertFit has 40 distributions, 30 high-quality graphical plots, 4 
technically correct goodness-of-fit tests, sample sizes of 100,000, interactive histograms, extensive support for simulation 
modeling, a distribution viewer, batch mode, comprehensive context-sensitive help, and a User’s Guide with 8 complete 
examples.” 

In the absence of historical data, ExpertFit provides only three alternative distributions to fit with a triad: the 
triangular, Weibull, and lognormal distributions. ExpertFit recommends Weibull and lognormal as the most useful 
for task duration times. 

4. Results of the Delphi Method 

The Delphi Method was used with the Orion Processing Team first, as a pathfinder. This process took a total of 
3 Vi months, which spanned method proposal, SME list development, task list preparation, training workshops for 
SMEs, distribution, reminders and follow up, ending with the analyses. The response rate for the initial project was 
63% among SMEs (see Table 1). Anecdotally, response rates depended upon whether experts were funded or not to 
work on the project, how familiar they were with the proposed ground processing design and vehicle design, and 
whether they were working exclusively on Constellation or were shared with Shuttle. 

Beyond the initial pathfinder, the Orion vehicle design matured over time and new tasks were identified or tasks 

were revised, updates to the task duration 
estimates were collected from the SMEs. 
More recently the Delphi method was also 
used for task duration estimation for 
processing the Lunar Spacecraft including 
Altair and Lunar Surface Systems. Table 1 
shows the response rates for these uses while 
Figure 3 shows the number of tasks estimated 
by different numbers of SMEs. 

Hundreds of tasks were estimated using the Delphi method. Varying numbers of SMEs provided inputs on the 
tasks, again based on their level and type of expertise. The team found that having at least 3 SMEs on any particular 
task was the minimum threshold. Handling dozens of SME spreadsheets was somewhat cumbersome. 


Table 1. Delphi Method Response Rates 



Responded 

Total 

Percent 

Responded 

First project 

42 

67 

63% 

Second project 

22 

47 

47% 

Third project 

26 

69 

38% 
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Feedback from SMEs 
indicated that about 3-5 
minutes per task was needed 
to provide most estimates 
from experience. At times 
research into historical data or 
into other factors was needed 
by SMEs to provide estimates 
which took much more time 
and was handled separately 
from the use of the Delphi 
Method. Generally, the SMEs 
at KSC learned the method 
quickly. The Delphi method 
has been used to develop 
input distributions for 
hundreds of tasks in several 
DES models at KSC. 

B. Probabilistic Event Variation 

During the identification of methods useful for verifying certain requirements, one requirement called for a 
contingency crew rescue within 24 hours 95% of the opportunities. The team responsible for this requirement 
identified four possible mission phases (launch phase, on-orbit phase, docked phase, prepare-for-landing phase) in 
which an abort event could occur and six possible zones (mid Pacific, U.S. West Coast, U.S. East Coast, 
Europe/Africa West Coast, South America West Coast, anywhere else) for which recovery assets had to be 
positioned or accessible for contingency water landings. The team estimated probabilities of aborts during each 
mission phase and conditional probabilities that a landing would occur in any of the six zones given that an abort 
occurred during a particular phase. They developed an analytical tool that used fixed probability estimates of aborts 
within mission phases to determine the overall probability of meeting the requirement. 

The DES Modeling & Analysis Center proposed extending the team’s model by varying the probability 
estimates that an abort event would occur in any given mission phase. This extension would account for part of the 
uncertainty in developing these estimates and would allow the requirement to be verified according to the planned 
Consumer’s risk approach, described below. Specifically, in the DES model, rather than modeling the probability 
that an event occurs as a decimal fraction (e.g. 0.85 for an 85% chance that an event occurs), the event would occur 
as in Equation (1) to account for the uncertainty in the 0.85 estimate and to vary the otherwise deterministic model 
output. Equation (1) is the sum of the deterministic portion, 0.85, with a normally distributed error term centered at 
mean 0.0 and standard deviation 0.005. An example of how this portion of the model could be implemented in 
Arena is shown in Figure 4. In the top portion of the code, an initialization sequence is executed which samples from 
the Normal distribution and stores values in the “Phase” and “Zone” variables, as shown in the “Assign” dialog box 
on the right side of Figure 4. The remainder of the code in Figure 4 shows the flow through the conditional 
probabilities. 

Consumer’s risk is the probability of stating that the planned system to meet a requirement will meet it, when in 
reality the system won’t meet it. In the Consumer’s risk approach, an experiment on a model is repeated. Each 
experiment results in a pass or fail outcome, i.e. either the crew were rescued in time (pass) or not (fail). A sample is 
taken and must fall below the rejection boundary, shown in Figure 5, to conclude that the rescue assets are 
sufficiently well positioned to meet the crew rescue requirement during contingency landings. The sample is used to 
make inferences on the performance of the underlying system. Figure 5 illustrates the sampling distributions, which 
depend only on a (also known as Producer’s risk or Type I error), the sample size a, and the proportion of items that 
fail in the sample to the total number of items,/?. Consumer’s risk is also known as or Type II error. 
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Though the team has not implemented this method to date, its novelty and accessibility were worth describing 
herein. 


This sequence initializes variables for each replcation, Then after the 
simulation finishes Its run, it outputs all the observed probabilities to 
the Excel spreadsheet, lar.xls. 
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Figure 4. Sample DES Code 


Rejection 

Boundary 


Sampling distribute 
of good design 



H 0 :p r p 0 = difference 
Where pis proportion of 
failures to opportunities 


To reduce p, 

• increase a, 

• increase sample size, or 

• increase hypothesized difference. 

p = Type II error, Consumer’s risk 
“a bad system Is accepted as good" 


a = Type I error, Producer’s risk 
“a good system is rejected as bad” 


Sampling distribution 
of bad design 



Figure 5. An Illustration of Consumer's Risk 
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C. Roll Up Method 

A large number of DES models have been developed by the team to answer questions across KSC’s facilities for 
processing a variety of spacecraft and flight hardware. At times parts of these models should be reused. One 
approach is to ‘copy’ elements of the model,’ as needed. But that would require maintaining these elements in two 
different models. Alternatively, elements could be reused by rolling up their values into a summary form. The team 
found no literature describing methods for rolling up, summarizing, or reusing parts of DES models in a less detailed 
form. Instead the team conducted experiments with sets of contrived detailed task sequences on how best to roll up 
and reuse them. Among these experiments was one with a 3x3 design, with independent variables and their levels as 
shown in Table 2. The number of tasks to be summarized was set to 4, 12, or 30 elements as a controlling variable. 

Table 2. Experiment Design for Roll Up Guidance 

Controlled Variable Independent Variable Independent Variable 

Number of Tasks Number of Replications Distributions 

4 30 Triangular 

12 200 Weibull 

30 1000 Lognormal 

The number of replications to execute the model was set to 30, 200, and 1000 to evaluate the effect of this 
independent variable. The fitted input distributions were all set to triangular, then to all Weibull, and finally all set to 
lognormal to evaluate the effect of this independent variable. 

All 27 combinations of this experiment design were executed. The output data sets of total processing time were 
fitted using ExpertFit. Based on the means, variances, and subjective interpretations of histograms of the outputs of 
the models, the independent variables that provided smooth, representative curves that were easy to fit by ExpertFit 
suggested using 200 replications with underlying lognormal distributions to summarize sequences of more detailed 
tasks. The lognormal distribution was considered better than triangular and Weibull input distributions for retaining 
skew. The lognormal is a useful task time distribution for sequences of tasks. The use of 200 replications was better 
than 30 or 1000 replications: thirty replications was not a sufficiently large number to capture the shape of the 
distribution of the outputs and ExpertFit produced poor fits. Using 1000 replications resulted in output distributions 
that appeared to be more symmetric and the mode of the distribution moved closer to the mean. Though the team is 
using this approach, much more research in this area should be conducted to discover improvements and ensure their 
validity and appropriateness. 

The team next evaluated the effect of adding constrained resources to the detailed model and the effect of 
parallel processing in the part of the model to be summarized. No discernible negative effects to this approach were 
uncovered, based on a few simple experiments using the 12-task models. Again, more research should be conducted 
to better evaluate this approach. Specifically, the theoretical basis for the number of replications and the basis of the 
lognormal recommendations should be analytically evaluated. And further experimentation to uncover when 
parallel processing and constrained resources dominate the effect of the task durations should be researched as well. 

The roll up approach was implemented by adding record statements to denote the beginning and ending times of 
the set of tasks to be summarized. The total time interval duration for 200 replications was output into Microsoft 
Excel. The data were then fed into ExpertFit. ExpertFit produced its best-fitting theoretical distributions, along with 
parameters and formulas suitable for use in Arena. These formulas were used in the less-detailed models as the 
representation of the delay duration for the several tasks summarized therein. 

IV. Conclusion 

0 Kennedy Space Center established the DES Modeling & Analysis Center in 2008 to inform decision making, aid 
in planning, minimize life cycle cost, and optimize facility and resource utilization. The DES center continues to 
reuse the Center’s substantial investment in DES over the years to develop and exploit new methods to best position 
KSC as the premier hub for human spaceflight and exploration. 
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DES 

GEM-FLO 

GOP 

KSC 

NGLT 

OSP 

SME 


Acronym List 
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Generic Environment for Modeling Future Launch Operations 

Ground Operations Project 
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Next Generation Launch Technology 

Orbital Space Plane Program 
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DES Analysis Methodology 


Discrete Event Simulation is a computer-based modeling technique for 
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complex and dynamic systems where the state of the system changes at 
discrete points in time and whose inputs may include random variables. 
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Modeling guidelines: 
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Examples of DES Successes at KSC 


♦ The Vehicle Assembly Building High Bay Assignments for Ares I 
and Ares V Vehicle Integration Activities 

♦ Launch Order and Spacing of Ares I and Ares V Combined 
missions 

♦ Ground Operations Preliminary Design Review (PDR) 
Requirements Compliance Analyses 

♦ Validation of Single String Processing Architecture to Save 
Maintenance Costs 

♦ Trade Studies for Optimum Facility Selection for Altair and 
Lunar Surface Systems Processing 

♦ Validation of Critical Path Processing Flows 

♦ Optimization of Launch Complex Operations 
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Launch-to-Launch Critical Path, Serial Work Hrs 


An Example of DES Used For 
Critical Path Analysis 




♦ Critical Path Requirement (879 hours) 

• Analysis revealed about 10% probability that 
the nominal timeline will require no more than 
934.4 hrs and about 90% probability that it 
will require no more than 965.72 hrs 

• Mean - 949.5 hrs 

• GOPD timeline single-point estimate 
duration - 855.18 hrs 

GOTAR DES Analysis Trend 
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The vertical bars show the effect of 
common cause variability in the task 
time estimates for the process flows, 
based on expert opinion. 

Opportunities exist to continue to 
reduce the total critical path 

• Incorporation of Interstage FOD net 

• LAS to CM offline integration 


AIM SpaceOps2010 Discrete Event Simulation 


5 


04/27/2010 


Three Extensions to DES Use at KSC 



♦ Delphi Method 

• Created by Rand Corporation and U.S. Air Force in late 1950s. 

• Extended here to collect subject matter expert task duration estimates 
used for fitting theoretical input distributions for DES models. Useful for 
new spacecraft designs where historical data do not exist. 

♦ Probabilistic Event Variation 

• Arena DES models have “decide” modules to choose alternative paths for 
the processing flow, typically using a fixed probability (e.g. 30% path A 
and 70% path B). The team extended Arena’s capability by varying these 
probabilities, such as with the Normal Distribution, because the probability 
is not known with certainty. 

♦ Roll Up Method 

• No literature was found for summarizing detailed DES models to be 
reused in less detailed DES models. 

• Some basic experiments were conducted to derive rules of thumb. These 
rules of thumb were used extensively to summarize and reuse portions of 
DES models in other models and are described here. 
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Delphi Method 



♦ The Delphi Method is a structured elicitation method for 
forecasting, estimating, and planning used with a group of 
subject matter experts in the absence of sufficient data. 
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Consumer’s Risk Approach to 
Requirement Acceptance 




A sample is drawn. The 
proportion of “pass” 
items to the total 
number of items is used 
to infer the proportion 
of the underlying 
population. The sample 
may be drawn from 
either a good or bad 
design. 
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Sampling distributio 
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Where p is proportion of 
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To reduce p, 
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• increase hypothesized difference 
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The proportion of simulation 
replications that “pass” was used to 
infer whether the system meets a 
requirement or not within the 
Consumer’s risk ceiling. 
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Probabilistic Event Variation 


♦ Conditional probabilities of crew rescue abort events, which depend on 
phase of flight and abort location. 

♦ Estimated by experts, subject to variance 

♦ Allows for consumer’s risk approach to requirement verification 
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Roll Up Method 


♦ A method was needed to reuse parts of more detailed DES 
models into parts of less detailed models. Experiments: 


Controlled Variables 

Independent Variables 

Number of Tasks 

Number of 
Replications 

Distribution 

4 

30 

Triangular 

12 

200 

Weibull 

30 

1000 

Lognormal 


♦ Rules of thumb: For each sequence of tasks that needed to be 
rolled up, 200 replications using lognormal as the underlying 
distribution were used to fit less detailed distributions. 

♦ More research should be done to evaluate when resources and 
parallel operations dominate the summary results and to 
investigate the theoretical basis for summarizing detailed DES 
models. 
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Example of Roll Up Method 


Using the GOTAR-07 DES Model, 200 output observations of 
the total time among the tasks within the Pad Operations 
processing segment were analyzed using Expert Fit to create 
rolled-up input distributions for the Surge Capacity model. 
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